FETI Coarse Problem Parallelization Strategies and Their Comparison

Most of computations (subdomain problems) appearing in FETI-type methods are purely local and therefore parallelizable without any data transfers. However, if we want to accelerate also dual actions, some communication is needed due to primal-dual transition. Distribution of primal matrices is quite straightforward. Each of cores works with local part associated with its subdomains. A natural effort using the massively parallel computers is to maximize the number of subdomains so that sizes of subdomain stiffness matrices are reduced which accelerates their factorization and subsequent pseudoinverse application, belonging to the most time consuming actions. On the other hand, a negative effect of that is an increase of the null space dimension and the number of Lagrange multipliers on subdomains interfaces, i.e. the dual dimension, so that the bottleneck of the TFETI method becomes the application of the projector onto the natural coarse space, especially its part called coarse problem solution. In this paper, we suggest and test different parallelization strategies of the coarse problem solution regarding to the improvements of the TFETI massively parallel implementation. Simultaneously we discuss some details of our FLLOP (Feti Light Layer on Petsc) implementation and demonstrate its performance on an engineering elastostatic benchmark of the car engine block up to almost 100 million DOFs. The best parallelization strategy based on the MUMPS was implemented into the multi-physical finite element based opensource code ELMER developed by CSC, Finland.